Automated Assessment and Standards Alignment of R Packages

class: center, middle, inverse, title-slide

# Automated Assessment and Standards Alignment of R Packages
### Mark Padgham & Noam Ross rOpenSci & EcoHealth Alliance Münster, Germany & New York, USA

---

class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# rOpenSci: Transforming science through open data, software & reproducibility

"*We help develop R packages for the sciences via community driven learning,
review and maintenance of contributed software in the R ecosystem*"

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# rOpenSci & Software Peer Review

"packages contributed by the community undergo a transparent, constructive, non
adversarial and open review process."

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# rOpenSci & Software Peer Review: Why?

- Feedback & improved software quality
- Support from rOpenSci
- Promotion of your software
- Cross-linking with other software
- Publications

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# rOpenSci & Software Peer Review: Scope

Current scope includes software for:

.left-column[

- data retrieval
- data extraction
- data munging
- data deposition
- data validation and testing
- workflow automation
- version control
- citation management and bibliometrics

]

.right-column[

- scientific software wrappers
- field and laboratory reproducibility tools
- database software bindings
- geospatial data
- text analysis

]

---
class: center, top
background-image: url(data:image/png;base64,#img/SloanLogo-1B-SMALL-Gold-Blue.png)
background-size: contain
background-position: 50% 50%

# Expanding Software Peer Review

.fontBIG[ Expand Scope to include Statistical Software ]

---
class: center, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# Expanding Software Peer Review

.fontBIG[ Expand Scope to include Statistical Software ]

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# Statistical Software Peer Review

## What is statistical software?

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# Statistical Software Peer Review

## What is statistical software?

### Methodology:

- Empirical collation of abstracts from:

- JSM conferences
    - *Journal of Statistical Software*
    - *Journal of Open Source Software*

- Identify key statistical terms and represent as relational network

- Analyse network structure & identify distinct clusters

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# Statistical Software Peer Review

## What is statistical software?

### Results:

.left-column[

- Bayesian and Monte Carlo
- Regression and Supervised Learning
- Dimensionality Reduction, Clustering, and Unsupervised Learning
- Exploratory Data Analysis (EDA) and Summary Statistics

]

.right-column[

- Time Series Analyses
- Machine Learning
- Spatial Analyses
- ~~Wrapper Packages~~
- ~~Network Analysis~~
- ~~Probability Distributions~~
- ~~Workflow Support~~

]

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# Statistical Software Peer Review

## Our approach

- General and category-specific standards

- Standards compliance documented within code

- Automated assessment tools

---
class: left, top
background-image: url(data:image/png;base64,#img/package-flow.png)
background-size: contain
background-position: 50% 50%

---
class: center, middle, inverse

# autotest

## github.com/ropensci-review-tools

## search: "ropensci autotest"

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# The 'autotest' package: Why?

## rOpenSci Software Peer Review

- A peer-review system integrating practices from software development and academic publishing

- 7 years experience, hundreds of packages, authors and reviewers

- Designed to improve software and expand a community of practices, via transparent, constructive, non-adversarial and open review

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# The 'autotest' package: Why?

## rOpenSci Software Peer Review

- Reviewers often the first "external" users of submitted packages

- Quickly uncover bugs, distracting reviews from higher-level focus on software quality

- A tool to help uncover bugs prior to submission should increase software quality

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# The 'autotest' package: How?

- A form of "mutation testing"

- Extracts documented example code to identify all inputs to, and outputs from,
  all functions

- Mutates those inputs

- Also matches input and output types to descriptions

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# The 'autotest' package

1/3: Set up a package to test:

```r
path <- file.path (tempdir (), "demo")
usethis::create_package (path)

fs::dir_tree (path)
```

```
## /tmp/RtmpRCSx84/demo
## ├── DESCRIPTION
## ├── NAMESPACE
## └── R
```

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# The 'autotest' package

2/3: Add a function:

```r
#' my_function
#'
#' @param x An input
#' @return Something else
#' @export
my_function <- function (x) {
 return (x + 1)
}
```

- save to `./R/myfn.R`

- run `roxygen::roxygenise()` to create documentation (`/man`) file

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# The 'autotest' package

3/3: `autotest` it:

```r
library (autotest)
a <- autotest_package (path, test = TRUE)
```

```
## 
## ── autotesting demo ──
## 
```

```r
class (a)
```

```
## [1] "autotest_package" "tbl_df"           "tbl"              "data.frame"
```

```r
dim (a)
```

```
## [1] 1 9
```

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# The 'autotest' package

3/3: `autotest` it:

```r
library (autotest)
a <- autotest_package (path, test = TRUE)
print (a)
```

```
## type test_name fn_name parameter parameter_type
## 1 warning fn_without_example my_function <NA> <NA>
## operation
## 1 Identify functions without documented examples
## content test yaml_hash
## 1 This function has no documented example TRUE <NA>
```

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# The 'autotest' package

3/3: `autotest` it:

```r
library (autotest)
a <- autotest_package (path, test = TRUE)
a$content
```

```
## [1] "This function has no documented example"
```

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# The 'autotest' package

```r
#' my_function
#'
#' @param x An input
#' @return Something else
#' @export
my_function <- function (x) {
 return (x + 1)
}
```

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# The 'autotest' package

```r
#' my_function
#'
#' @param x An input
#' @return Something else
#' @export
#' @examples
#' y <- my_function (x = 1)
my_function <- function (x) {
 return (x + 1)
}
```

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# The 'autotest' package

```r
library (autotest)
a <- autotest_package (path, test = TRUE)
```

```
## 
## ── autotesting demo
## ✔ [1 / 1]: myfunction
```

```r
a$content
```

```
## [1] Parameter [x] is not specified as integer, yet
##     only used as such; please use '1L' for integer,
##     or 1.0 for non-integer values.
## [2] Parameter [x] of function [my_function] is only used
##     as a single numeric value, but responds to vectors of
##     length > 1
```

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# The 'autotest' package

```r
#' my_function
#'
#' @param x An input
#' @return Something else
#' @export
#' @examples
#' y <- my_function (x = 1)
my_function <- function (x) {
 return (x + 1)
}
```

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# The 'autotest' package

```r
#' my_function
#'
#' @param x An input
#' @return Something else
#' @export
#' @examples
#' y <- my_function (x = 1L)
my_function <- function (x) {
 checkmate::assertInt (x)
 return (x + 1)
}
```

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# The 'autotest' package

```r
library (autotest)
a <- autotest_package (path, test = TRUE)
```

```
## 
## ── autotesting demo
## ✔ [1 / 1]: myfunction
```

```r
a$content
```

```
## [1] Parameter [x] permits unrestricted integer inputs,
##     yet does not document this; please add 'unrestricted'
##     to parameter description.
```

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# The 'autotest' package

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# The 'autotest' package

```r
#' my_function
#'
#' @param x An unrestricted integer input
#' @return Something else
#' @export
#' @examples
#' y <- my_function (x = 1L)
my_function <- function (x) {
 checkmate::assertInt (x)
 return (x + 1)
}
```

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# The 'autotest' package

```r
library (autotest)
a <- autotest_package (path, test = TRUE)
```

```
## 
## ── autotesting demo
## ✔ [1 / 1]: myfunction
```

```r
a
## NULL
```

### When `autotest` works, it should do nothing!

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# The 'autotest' package

## Which tests are run? `test = FALSE`

```r
a <- autotest_package (path, test = FALSE)
a
```

```
## # A tibble: 9 x 9
## type test_name fn_name parameter parameter_type operation content test 
## <chr> <chr> <chr> <chr> <chr> <chr> <chr> <lgl>
## 1 dummy int_range my_func… x single integer Ascertain … Should e… TRUE 
## 2 dummy int_as_nu… my_func… x single integer Integer va… (Should … TRUE 
## 3 dummy single_pa… my_func… x single integer Length 2 v… Should t… TRUE 
## 4 dummy return_su… my_func… (return … (return objec… Check that… <NA> TRUE 
## 5 dummy return_va… my_func… (return … (return objec… Check that… <NA> TRUE 
## 6 dummy return_de… my_func… (return … (return objec… Check whet… <NA> TRUE 
## 7 dummy return_cl… my_func… (return … (return objec… Compare cl… <NA> TRUE 
## 8 dummy par_is_do… my_func… x <NA> Check that… <NA> TRUE 
## 9 dummy par_match… my_func… x <NA> Check that… <NA> TRUE 
## # … with 1 more variable: yaml_hash <chr>
```

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# The 'autotest' package

## Which tests are run? `test = FALSE`

```r
a <- autotest_package (path, test = FALSE)
a$test_name
```

```
## [1] "int_range"                  "int_as_numeric"            
## [3] "single_par_as_length_2"     "return_successful"         
## [5] "return_val_described"       "return_desc_includes_class"
## [7] "return_class_matches_desc"  "par_is_documented"         
## [9] "par_matches_docs"
```

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# The 'autotest' package

## Which tests are run? `test = FALSE`

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# The 'autotest' package

## Which tests are run? `test = FALSE`

```r
#' my_function
#'
#' @param x An unrestricted integer input
#' @return Something else
#' @export
#' @examples
#' y <- my_function (x = 1.)
my_function <- function (x) {
 checkmate::assertNumber (x)
 return (x + 1)
}
```

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# The 'autotest' package

## Which tests are run? `test = FALSE`

```r
a <- autotest_package (path, test = FALSE)
a$test_name
```

```
## [1] "double_is_int"              "trivial_noise"             
## [3] "single_par_as_length_2"     "return_successful"         
## [5] "return_val_described"       "return_desc_includes_class"
## [7] "return_class_matches_desc"  "par_is_documented"         
## [9] "par_matches_docs"
```

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# The 'autotest' package

A real example: `autotest`ing the `stats` package

```r
a <- autotest_package (package = "stats",
 functions = "var",
 test = FALSE) # default
```

- `package` argument can be name of an installed package, or path to local source directory

- `functions` argument can restrict tests to specified functions only

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# The 'autotest' package

A real example: `autotest`ing the `stats` package

```r
a <- autotest_package (package = "stats",
 functions = "var",
 test = FALSE) # default
```

```
## 
## ── autotesting stats
## ✔ [1 / 4]: var
## ✔ [2 / 4]: cor
## ✔ [3 / 4]: cov
## ✔ [4 / 4]: cov
```

```r
dim (a)
```

```
## [1] 150   9
```

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# The 'autotest' package

A real example: `autotest`ing the `stats` package

```r
a <- autotest_package (package = "stats",
 functions = "var",
 test = TRUE) # takes 20-30 seconds
```

```
## 
## ── autotesting stats
## ✔ [1 / 4]: var
## ✔ [2 / 4]: cor
## ✔ [3 / 4]: cov
## ✔ [4 / 4]: cov
```

```r
dim (a)
```

```
## [1] 15  9
```

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# The 'autotest' package

A real example: `autotest`ing the `stats` package

```
## # A tibble: 15 x 9
## type test_name fn_name parameter parameter_type operation content test 
## <chr> <chr> <chr> <chr> <chr> <chr> <chr> <lgl>
## 1 warni… par_is_de… var use <NA> Check tha… "Example… TRUE 
## 2 warni… par_is_de… cov y <NA> Check tha… "Example… TRUE 
## 3 diagn… single_ch… cor use single charac… upper-cas… "is case… TRUE 
## 4 diagn… single_ch… cor method single charac… upper-cas… "is case… TRUE 
## 5 diagn… single_ch… cor use single charac… upper-cas… "is case… TRUE 
## 6 diagn… single_ch… cor method single charac… upper-cas… "is case… TRUE 
## 7 diagn… single_ch… cov use single charac… upper-cas… "is case… TRUE 
## 8 diagn… single_ch… cov method single charac… upper-cas… "is case… TRUE 
## 9 diagn… single_ch… cov use single charac… upper-cas… "is case… TRUE 
## 10 diagn… single_ch… cov method single charac… upper-cas… "is case… TRUE 
## 11 diagn… single_ch… cov use single charac… upper-cas… "is case… TRUE 
## 12 diagn… single_ch… cov method single charac… upper-cas… "is case… TRUE 
## 13 diagn… vector_to… var x vector Convert v… "Functio… TRUE 
## 14 diagn… vector_to… var x vector Convert v… "Functio… TRUE 
## 15 diagn… vector_to… var y vector Convert v… "Functio… TRUE 
## # … with 1 more variable: yaml_hash <chr>
```

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# The 'autotest' package

A real example: `autotest`ing the `stats` package

- Use `DT` package to inspect results!

```r
a <- autotest_package (package = "stats",
 functions = "var",
 test = TRUE) # takes 20-30 seconds
DT::datatable (a)
```

---
class: left, top
background-image: url(data:image/png;base64,#img/stats-var-TRUE.png)
background-size: contain
background-position: 50% 50%

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# The 'autotest' package

## Which tests are run? `test = FALSE`

```r
a <- autotest_package (package = "stats",
 functions = "var",
 test = FALSE)
dim (a)
```

```
## [1] 150   9
```

```r
DT::datatable (a)
```

---
class: left, top
background-image: url(data:image/png;base64,#img/stats-var-FALSE1.png)
background-size: contain
background-position: 50% 50%

---
class: left, top
background-image: url(data:image/png;base64,#img/stats-var-FALSE2.png)
background-size: contain
background-position: 50% 50%

---
class: left, top
background-image: url(data:image/png;base64,#img/stats-var-FALSE3.png)
background-size: contain
background-position: 50% 50%

---
class: left, top
background-image: url(data:image/png;base64,#img/stats-var-FALSE4.png)
background-size: contain
background-position: 50% 50%

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# The 'autotest' package

A real example: `autotest`ing the `stats` package

```r
a <- autotest_package (package = "stats",
 functions = "var",
 test = TRUE)
```

- `stats` package is algorithmially robust

- But `autotest` reveals a few gaps in documentation!

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# Statistical Software Peer Review

## Our approach

- General and category-specific standards

- Standards compliance documented within code

- Automated assessment tools

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# Statistical Software Peer Review

## Our approach

- General and category-specific standards

- Standards compliance documented within code

- ~~Automated assessment tools~~

---
class: center, middle, inverse

# Statistical Standards

## Online Book Contains all General and Category-Specific Standards

`stats-devguide.ropensci.org`

---
class: left, top
background-image: url(data:image/png;base64,#img/devguide-stds-G1.png)
background-size: contain
background-position: 50% 50%

---
class: left, top
background-image: url(data:image/png;base64,#img/devguide-stds-G2.png)
background-size: contain
background-position: 50% 50%

---
class: left, top
background-image: url(data:image/png;base64,#img/devguide-stds-G3.png)
background-size: contain
background-position: 50% 50%

---
class: left, top
background-image: url(data:image/png;base64,#img/devguide-stds-G4.png)
background-size: contain
background-position: 50% 50%

---
class: left, top
background-image: url(data:image/png;base64,#img/devguide-stds-BS1.png)
background-size: contain
background-position: 50% 50%

---
class: left, top
background-image: url(data:image/png;base64,#img/devguide-stds-BS2.png)
background-size: contain
background-position: 50% 50%

---
class: left, top
background-image: url(data:image/png;base64,#img/devguide-stds-BS3.png)
background-size: contain
background-position: 50% 50%

---
class: center, middle, inverse

# Statistical Standards

## 40 General Standards & 265 Category-Specific Standards in 7 Categories

## Four New Categories in Progress

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# Statistical Software Peer Review

## Our approach

- General and category-specific standards

- Standards compliance documented within code

- ~~Automated assessment tools~~

---
class: left, top
background-image: url(data:image/png;base64,#img/package-flow.png)
background-size: contain
background-position: 50% 50%

---
class: center, middle, inverse

# The 'srr' package

## `srr` = "Software Review Roclets"

## github.com/ropensci-review-tools

## search: "ropensci srr"

(A "roclet" is a documentation engine, or "doclet", for the **R** language.)

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# The 'srr' package: Why?

- Software must comply with detailed, category-specific standards

- `srr` allows standards compliance to be documented within the software itself

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# The 'srr' package

## documentation in R with 'roxygen2'

```r
#' Function description
#'
#' @rdtitle NewFunction
#' @param x First input
#' @param y Second input
#' @return Something different
#' @export
myfunction <- function (x, y) {
 # ...
}
```

Run `roxygen2::roxygenise()` to turn that into `/man` file `NewFucntion.Rd`

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# The 'srr' package

## documentation in R with 'roxygen2'

- `roxygen2` is extensible, and allows custom "roclets"

- Everything following `@` is a "tag"

- Each tag has a "roclet"

- Each roclet has a generic `process` method, and an `output` method to
  output information to the `.Rd` file

- Tags with `output = NULL` do not appear in package documentation

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# The 'srr' package

- `srr` introduces 3 new tags for documenting standards:

- `@srrstats` for adherence to standards

- `@srrstatsNA` for non-applicable standards

- `@srrstatsTODO` for standards yet to be addressed

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# The 'srr' package

- The `srr_stats_roxygen()` function dumps full text of all standards in file
  `R/srr-stats-standards.R`

- All are initially tagged `@srrstatsTODO`

- Standards are addressed by changing tag to `@srrstats` and moving to location
  within code where standard is addressed

- `srr_report()` function generates a `.html` report of standards compliance

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# The 'srr' package

```r
#' Function description
#'
#' @rdtitle NewFunction
#' @param x First input
#' @param y Second input
#' @return Something different
#' @export
myfunction <- function (x, y) {
 # ...
}
```

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# The 'srr' package

```r
#' Function description
#'
#' @rdtitle NewFunction
#' @param x First input
#' @param y Second input
#' @return Something different
#' @export
#'
#' @srrstats {G1.1} This standard is addressed here through ...
myfunction <- function (x, y) {
 # ...
}
```

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# The 'srr' package

```r
library (srr)
path <- srr_stats_pkg_skeleton()
roxygen2::roxygenise (path)
```

```
## ℹ Loading demo
## Re-compiling demo
## ─────────────────────────────────────── rOpenSci Statistical Software Standards ──────────────────────────────────────
## 
## ── @srrstats standards:
##   * [G1.1, G1.2, G1.3, G2.0, G2.1] in function 'test_fn()' on line#11 of file [R/test.R]
##   * [RE2.2] on line#2 of file [tests/testthat/test-a.R]
##   * [G2.3] in function 'test()' on line#6 of file [src/cpptest.cpp]
##   * [G1.4] on line#17 of file [./README.Rmd]
## 
## ── @srrstatsNA standards:
##   * [RE3.3] on line#5 of file [R/srr-stats-standards.R]
## 
## ── @srrstatsTODO standards:
##   * [RE4.4] on line#14 of file [R/srr-stats-standards.R]
##   * [RE1.1] on line#11 of file [R/test.R]
##   * [G1.5] on line#17 of file [./README.Rmd]
## ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
```

---
class: left, top
background-image: url(data:image/png;base64,#img/icon_lettering_color_large-faded.png)
background-size: contain
background-position: 50% 50%

# Statistical Software Peer Review

- Now accepting submissions

- Use of `autotest` will enable reviews to better focus on substantive issues of software design and quality

- `srr` provides a comprehensive method for documenting standards
  compliance within code itself

- Associated `html` reports facilitate rapid review of standards compliance

- Result should be high quality statistical software with open and demonstrable compliance with standards